AI Realtime Voice GuideConfigure API key, voice, and role; start/stop realtime voice dialog

1. Overview

Note: Keep the robot online with a working network. An invalid API Key or unavailable service may cause start failure or broken dialog.

2. Entry & Page Layout

1. Entry: On the navigation home page, tap AI Realtime Voice to open this feature.
2. Page layout:
   • Top status: Running / Stopped, camera On/Off, and the Model menu (Qwen / OpenAI / Doubao)
   • Latest dialogue: current question and AI answer
   • Control area: Start, Stop, Settings (role, key, voice), Knowledge Base, Live Assistant (Qwen only)

Figure 1: Navigation entry (tap “AI Realtime Voice”)

Figure 2: Main page (status, dialogue, controls)

2.1 Switch model

1. On the status card, tap Model.
2. Choose Alibaba Cloud (Qwen) or Doubao in China, or OpenAI (GPT) outside China.
3. Confirm. The running realtime voice stops first, then the matching model is used.
4. After switching to Qwen or GPT, enter that side’s API key under Settings, and that side’s knowledge base under Knowledge Base. Do not put a Qwen key on OpenAI, or an OpenAI key on Qwen.
5. After switching to Doubao, open Settings and fill in the API key, search key, voice, and role prompt as in section 11. Then save and start.

Networks in mainland China usually cannot reach OpenAI. Use Qwen or Doubao there.

Figure 2-1: Switch the model and select OpenAI (GPT), then enter the key in Settings

3. Configure API Key / Voice / Role

Complete configuration before first use. Tap Settings in the control area.

• Model is Alibaba Cloud (Qwen): enter a Qwen API Key. See China: Qwen API Key.
• Model is OpenAI (GPT): enter an OpenAI API Key. See Outside China: OpenAI API Key. The key must match the selected model.
• Model is Doubao: enter the Doubao API key. Add the search key for built-in web search. Voices are only vivi, Xiaohe, Yunzhou, and Xiaotian; Xiaohe is the default. See Doubao.

3.1 API Key

1. Enter a valid API Key in the API Key field.
2. Use the eye icon to show/hide the key.
3. Tap Save for changes to take effect.

Keep the key private; do not share it with unrelated people.

3.2 Voice

1. Tap the Voice row to open the voice picker.
2. Select a voice (default recommendation: Tina).
3. Confirm, return to Settings, then tap Save.

Chinese, dialects, and multilingual voices (e.g. Sichuan, Cantonese, English) are supported.

3.3 Role prompt

1. Enter a role prompt in the Role prompt field.
2. Recommended: robot identity, reply style, and whether to prefer action/move tools.
3. Tap Save.

Example (adapt to your site):

You are a humanoid robot assistant of Chengdu Xinfengyun Technology Co., Ltd. Your name is Xiao Feng. You can control your own movement and performance actions via tools.
Rules:
1. When the user asks what you can do or which actions are available, you must call list_actions.
2. When the user asks to move forward/backward, turn left/right, or stop, you must call the corresponding move tool — do not only agree verbally.
3. When the user asks for wave, salute, dance, heart gesture, handshake, etc., you must call execute_action.
4. Keep answers concise and friendly in English. After an action, confirm in one short sentence.

Figure 3: Settings — API Key, Voice, and Role prompt

Figure 4: Voice picker

4. Knowledge Base

This chapter is for company documents on Qwen and GPT. For Doubao weather, news, and the current time, enter the search key under Settings. Steps are in section 11.

The knowledge base lets realtime voice answer from your documents instead of only the model’s built-in knowledge. Open it from Knowledge Base in the control area. After saving, stop realtime voice and start it again before the new base is used.

4.1 Pick the model first

Note: Switch between Qwen and GPT on this page and the knowledge-base form changes with it. Networks in mainland China usually cannot reach OpenAI, so choose Qwen or Doubao there. Do not paste an OpenAI key or vector store ID into Qwen, or the three Qwen IDs into OpenAI. Doubao keys are separate; see Doubao.

4.2 Alibaba Cloud knowledge base (Qwen selected)

Switch the model to Alibaba Cloud (Qwen), then open Knowledge Base. Fill in the four fields below and tap Save. Workspace, Agent ID, and Host all come from the Alibaba Cloud Model Studio console, and they must use the same region as the API Key. Use China (Beijing).

Figure 5: Domestic knowledge base settings (Workspace / Agent ID / Host / Tool definition)

Get the three IDs in Model Studio:

1. Sign in to the Model Studio console and select China (Beijing). Create the API Key in section 9 in that same region.
2. Workspace: open the current workspace details at the bottom-left of the console, or open workspace management, and copy the workspace ID. Docs: Manage workspaces.
3. Create a knowledge base, upload scripts or product documents, and wait until parsing finishes. Docs: Create and use a knowledge base.
4. Create and publish a knowledge Q&A service, and attach that knowledge base. Copy the application ID into Agent ID. An unpublished service cannot be called. Docs: Knowledge Q&A.
5. Host for Beijing is {workspaceId}.cn-beijing.maas.aliyuncs.com. If the workspace ID is llm-abc123, the host is llm-abc123.cn-beijing.maas.aliyuncs.com.
6. Write the tool definition and save. This text decides when the model calls knowledge_search. Update it whenever you switch knowledge bases.

Tool definition example (rewrite it for your documents):

Search this venue knowledge base (exhibit introductions, product specs, visit routes). When the user asks about exhibits, the company, or on-site explanations, you must call this tool first and must not answer from memory. Do not use it for robot motion, navigation, weather, news, or the current time.

Timeout, result count, and minimum score are not edited on this page. Existing values in the config file are kept. If loading fails, the page uses defaults — check that the robot is online and open the dialog again.

4.3 OpenAI knowledge base (GPT selected)

Switch the model to OpenAI (GPT), then tap Knowledge Base. The dialog has two fields: Knowledge ID and Description. The ID is required. After you save, timeout and result-count settings stay as they are. The new base is used the next time realtime voice starts.

Figure 6: Overseas knowledge base settings (Knowledge ID / Description)

1. Finish Outside China: OpenAI API Key, switch the model to OpenAI (GPT), and save that key under Settings.
2. Sign in at OpenAI Platform and open Vector stores:

3. Create a vector store and name it, for example “Venue guide”.
4. Upload PDF, Word, or TXT files and wait until the status is Completed. Do not save the ID to the robot while files are still processing.
5. Copy the vector store ID. It usually starts with vs_. Paste it into Knowledge ID.
6. In Description, state what the files cover and when knowledge_search must be called. Change this text when you switch stores, or the model will not know when to search.
7. Save, then stop and start realtime voice.

Description example:

Search the venue guide knowledge base (exhibits, product specs, visit routes). When the user asks about exhibits, the company, or on-site explanations, you must call knowledge_search first. Do not use it for robot motion, navigation, weather, news, or the current time.

Official guide: OpenAI Retrieval / Vector stores. If Vector stores is not in the sidebar, look under Storage.

5. Live Assistant

Qwen can open Live Assistant. It connects the robot to a Douyin live room, warms up the room, reads comments, and welcomes visitors. Warm-up interval, warm-up scripts, and welcome scripts can be customized. Open it from Live Assistant in the control area. After saving, stop realtime voice and start it again before the change is used. Switch the model to Qwen first.

Note: Enter the room ID from the Douyin website, not a stream push URL and not the share text from the app. A room ID is required when the assistant is enabled.

5.1 Room ID

1. On a computer browser, open the Douyin live room that is on air. The address looks like:

2. The last segment (123456789 in the example) is the room ID. You can also paste the full link; the ID is extracted when you save.
3. Turn on Enable live assistant.
4. Do not type the room URL yourself. On save it is written as https://live.douyin.com/{room ID}.
5. The room ID may contain only letters, numbers, underscores, and hyphens. Save is blocked if the assistant is on and the ID is empty.

Figure 7: Live Assistant settings (enable / room ID / prompt switches)

5.2 Prompts

Each row has a switch and an Edit button. Turning the switch off stops that announcement. Tap Edit to open a second dialog, tap Confirm, then return and tap Save.

Welcome example (one line each):

Welcome {username} to the live room. I’m Xiao Feng — tap follow if you like it.
Welcome {username}. Come in, look around, and ask anything.

Danmaku reply example (one instruction):

You are the live assistant. Someone in the room asked: {content}. Reply in one or two spoken sentences. Do not read this instruction aloud, and do not call tools.

Figure 8: Edit a prompt (one line each; confirm, then save on the settings page)

Figure 8-1: Editing the AI warm-up prompt. Change the interval and the prompt text. Screenshot shows the Chinese UI.

Welcome, gift, and like intervals stay preset. Warm-up lines default to every 180 seconds, and AI warm-up defaults to 120 seconds of silence. Change those two in their Edit dialogs. The editor is no longer covered by the page behind it. If loading fails, default prompts are used — check that the robot is online and open the dialog again.

5.3 Danmaku actions

The switch at the bottom, Danmaku actions, is off by default. Turn it on to let Live Assistant trigger robot actions from comments. Leave it off for voice only. Save, then stop and start realtime voice.

6. Camera Switch

1. In the top status area, find the Camera On / Off switch.
2. On: enable camera streaming so replies can use visual context.
3. Off: voice-only. If you turn it off mid-session, the AI continues based on the last frame it saw.

Switch changes save immediately; you do not need to open Settings again.

Figure 9: Top camera switch

7. Start & Stop

1. Start: After saving a valid API Key, tap Start. Wait a few seconds until status shows Running.
2. Talk: Speak to the robot. The Latest dialogue area shows Question and Answer.
3. Stop: Tap Stop; status becomes Stopped when the service ends.

Tip: Buttons show a loading state during start/stop — avoid rapid repeated taps. If start fails, check scripts, network, and API Key.

Figure 10: Controls (Start / Stop / Settings)

8. Dialogue Display & FAQ

8.1 Latest dialogue

The middle “Latest dialogue” area shows the current turn:

• Question: speech recognition result
• Answer: AI realtime voice reply text

Before any dialog, it shows “No dialogue yet”.

Figure 11: Latest dialogue

8.2 FAQ

9. China: Qwen API Key

Use this flow when the model is Alibaba Cloud (Qwen). The dialog key is an Alibaba Cloud Model Studio (Qwen) API Key, and the knowledge base is a Model Studio knowledge base on the same account (see Knowledge Base). In China, Qwen has been upgraded from 3.5 to 3.8: lower cost, steadier answers, and it can speak Mongolian. Follow the steps below.

Step 1: Register an Alibaba Cloud account and complete identity verification

1. Open the Alibaba Cloud registration page:

2. Complete registration as prompted.
3. After login, complete identity verification (personal or enterprise). Without verification, related services usually cannot be enabled. Verification entry:

Figure 12: Register and complete identity verification

Step 2: Create an API Key

1. Open the Alibaba Cloud Model Studio (Bailian) console API Key page:

2. On first visit, enable Bailian / model calling and accept the agreements.
3. Select region (top-right). Recommended: China (Beijing).
4. Open API Key management and click Create API Key.
5. In the dialog, suggested settings:
   • Workspace: default workspace
   • Description: optional, e.g. “AI Realtime Voice”
   • Permissions: recommend “All”
6. After confirm, copy and store the full key immediately (usually starts with sk-). The full plaintext is usually not shown again after closing; if lost, reset or create a new one.

Figure 13: API Key management page

Figure 14: Create API Key

Figure 15: Copy and save the API Key

Warning: An API Key is a credential. Do not leak it, share screenshots, or commit it to public repos. Paste and save it under AI Realtime Voice → Settings in this app.

Official help: How to get an API Key (Alibaba Cloud Help Center). After you have the key, configure the Alibaba Cloud knowledge base in section 4.

10. Outside China: OpenAI API Key

Use this flow when the model is OpenAI (GPT). Both the dialog key and the knowledge base come from OpenAI: paste the key under Settings, and paste the vector store ID under Knowledge Base (see 4.3 OpenAI knowledge base). Networks in mainland China usually cannot complete this signup or call the API — use the Qwen key in section 9 instead.

Step 1: Create an OpenAI Platform account

1. Open the signup page (or sign in at platform.openai.com if you already have an account):

2. Register with email, or with Google, Microsoft, or Apple.
3. Verify your email. If the page asks for a phone number, use one that can receive SMS or a call.
4. The first login creates an Organization. Keep the key and the vector store in that same organization.

Step 2: Add billing and credits

A key with no credit balance cannot run realtime voice or knowledge search. Add a payment method and prepaid credit first.

1. Open Billing and add a credit card or another method the page accepts.
2. Add prepaid credit. Realtime voice and vector search are both charged from that balance.
3. Set a usage limit if you want to cap unexpected spend. Current prices are on Pricing.

Note: This is the OpenAI API bill, not a ChatGPT Plus subscription. A ChatGPT web subscription does not replace this API key.

Step 3: Create an API key

1. Click Create new secret key.
2. Suggested settings:
   • Name: for example AI Realtime Voice
   • Project: the current project (Default is fine). Create the vector store in the same project, or this key cannot read it.
   • Permissions: All. If you use a restricted key, it still needs model calls, files, and vector stores.
3. The full secret is shown only once. Copy it immediately. It usually starts with sk- or sk-proj-. After you close the dialog it cannot be viewed again; create a new key if it is lost.
4. Open AI Realtime Voice, switch the model to OpenAI (GPT), open Settings, paste the key, and save.
5. Stop and start realtime voice so the new key is used.

Warning: The API key is a credential. Do not leak it, share screenshots, or commit it to a public repository. Revoke unused keys on the API keys page.

Official page: API keys. To create a knowledge base and paste the vs_ ID, see 4.3 OpenAI knowledge base.

11. Doubao

Follow this section when the model is Doubao. Doubao uses Volcengine full-duplex realtime voice. It can listen and speak at the same time, barge in, run actions, movement, and navigation, and sing on request. Fill in the search key before asking for weather, news, or the current time.

11.1 Configure and start

1. Set the model to Doubao. A running session stops first.
2. Open Settings.
3. Enter the API key. How to get it is in 11.2. This is the Volcengine Doubao Speech API key, not an App ID and not an access token.
4. To answer weather, news, or the time, enter the search key (search_api_key). How to get it is in 11.3. You can leave it empty and still start a conversation; live public facts will wait until the key is saved.
5. Choose a voice. The choices are vivi, Xiaohe, Yunzhou, and Xiaotian. The default is Xiaohe.
6. Fill in the role prompt. The Chinese Xiaofeng prompt can stay. State that it should call action, movement, and navigation.
7. Tap Save, then Start.

Note: The API key and the search key are stored for Doubao. If realtime voice is already running, stop it and start again after saving.

11.2 Get the API key

Enable end-to-end realtime voice on Volcengine and copy the API key.

1. Sign in to Volcengine with the company account. Register first if you do not have one.
2. Open Doubao Speech and enable end-to-end realtime voice (full duplex). Product notes: full-duplex documentation.
3. Create an app and copy the API Key. Paste it into API key on this page. Do not paste an App ID or an access token.
4. Return to AI Realtime Voice, keep the model on Doubao, paste the key into API key, save, and start again. Continue below if you also need web search.

If the console shows a key only once, copy it immediately. Do not share screenshots or put the key in a public file.

11.3 Get the search key (search_api_key)

The Search key field in Settings is search_api_key. It is the Volcengine fusion information search API key, and it lets Doubao answer weather, news, and the current time. Do not paste the Doubao Speech API key here.

1. Sign in with the same Volcengine company account used for Doubao Speech.
2. Open the fusion-search API key page above. If the service is not enabled, enable fusion information search first.
3. Create an API key, or copy an existing one.
4. Return to AI Realtime Voice, keep the model on Doubao, open Settings, and paste it into Search key.
5. Tap Save. If realtime voice is already running, stop it and start again so this session can search.

Note: An empty search key still lets the dialog start. That session just will not look up live information. Do not share screenshots of either key, and do not put them in a public file.

11.4 Singing, actions, and navigation

• Singing: after startup, ask for a song and Doubao sings it.
• Actions and movement: say an action name, or say walk forward, walk back, turn left, or turn right. Asking what it can do lists up to 100 actions.
• Navigation: with realtime voice running and navigation localized, say a tour name or keyword to start that tour.